Edge Detection: 2D Operators

Intro

Let’s take what we’ve learned in one dimension and apply it to a two dimensional image.

Remember how edges are related to gradients; we developed some operators that you can apply to gradients and we discussed first applying filters to smooth out the noise and in what order that can be done to save operations.

Derivative of Gaussian Filter 2D

In 2D it’s not just a derivative but also the direction of the derivative.

Suppose we want to take the derivative in the x direction.

\[(I\otimes g) \otimes h_x = I \otimes (g \otimes h_x)\]

What this sub x is implying is that we are only applying the derivative in the x direction. This could be the Sobel operator or any other operator that requires a derivative. It’s a small filter taking a derivative. g is the gaussian function and the associative property allows for moving the operation around.

This gives us the derivative of the gaussian smoothed image from before. It’s preferable to do it this way, but why?

Quiz: Smoothing

Why is it preferable to apply \(h\) to the smoothing function \(g\) and apply the result to the image?

Is the horse dead yet? The smoothing function \(g\) is generally smaller than the image. Applying \(h\) to the smaller function requires fewer operations that then don’t need to be repeated.

Effect of \(\sigma\) on Derivatives

Smoothed Gaussian

Smoothed Gaussian

How big of a gaussian should we use? What effect does a changing spread have on the derivative? As your \(\sigma\) grows your smoothing effect will magnify; the same effect holds true when computing derivatives, enhancing the magnitude of the derivative as it grows in space.

What effect does this have in practice? Smaller values will capture finer features as edges and larger values will filter out to only large features.

Canny Edge Operator

How do we actually find the edges?

  1. Smoothing derivatives to suppress noise and compute the gradient.
  2. Threshold the output to find regions of ‘significant’ gradient.
  3. ‘Thin’ the results to get localized edge pixels. (Will talk more about this later; this reduces thick edges to thin contour.)

This operation, the Canny operator, works as follows:

  1. Filters with the derivative of a Gaussian.
  2. Find the magnitude and orientation of the gradient.
  3. Perform non-maximum suppression where you thin multi-pixel wide ‘ridges’ down to a single pixels width. (Thinning)
  4. Linking and thresholding. Define two thresholds, a high and a low, and use the high threshold to start edge curves and the low threshold to continue them.

In python this is called with

import cv2
im = cv2.imread('image')
cv2.Canny(im,...)

Canny Edge Detector

So, let’s walk through Canny edge detection in practice.

Step 1: Image

Given some image, we would like to detect edges using the method that Canny created.

Step 2: Gradient

Although this looks like edges to a person, it’s simply the magnitude of the gradient. It needs to be worked on prior to being useful.

Step 3: Thresholding

Getting rid of anything in the image that is above / below a threshold allows us to quickly and easily discard things that most likely are not edges. Lena got a haircut…

Step 4: Thinning

Thinning allows representing large and wide bands of ‘edge’ as thin and simple contour lines by checking for local maxima along the gradient direction.

Along the direction of the gradient, as seen in the images to the right, there will be a local maxima somewhere on the line of the gradient direction. The ‘thinned’ edge is represented solely by that local maxima. This can require subpixel accuracy when interpolating!

Step 5: Canny Threshold Hysteresis 1

  1. Apply a high threshold to detect strong pixels.
  2. Link those strong edge pixels to form strong edges.
  3. Apply a low threshold to find weak but plausible edge pixels.
  4. Extend the strong edges to follow weak edge pixels.

This assumes that noisy edges that are close to strong edges are likely edges.

For Your Eyes Only Demo

This code block reads in the frizzy and froomer images, uses a Canny filter to compute the edges in the image, and then uses a boolean operation to detect where edges in the two images overlap.

import cv2

# Helper function to show images.

from cs6476helpers import imshow

# Read in the data

frizzy = cv2.imread("frizzy.png")
froomer = cv2.imread("froomer.png")

# Show off the original images

imshow(frizzy)
imshow(froomer)

# Calculate the edges using the Canny filter
#  Note that 10 and 20 for thresholds here are arbitrary.

frizzy_edges = cv2.Canny(frizzy, 10, 20)
froomer_edges = cv2.Canny(froomer, 10, 20)

# Show off the edges

imshow(frizzy_edges)
imshow(froomer_edges)

# Overlay the edges

shared_edges = frizzy_edges & froomer_edges

# And show off where they overlap!

imshow(shared_edges)

Frizzy

Edges

Froomer

Edges

Shared Edges

Quiz: For Your Eyes Only

What is the secret code in the image above? (cue James Bond music…)

Canny Results

It’s really hard to know when an edge image is good. What will you use them for?

The Canny operator tends to be better at pulling out edges you want to use for future processing.

What size of kernel should we use in a Canny filter?

Large \(\sigma\) detects large edges and small detects small.

Quiz: Canny Edge

Is the Canny operator sensitive to noise? This is mostly false and is dependent on the \(\sigma\) chosen for the smoothing operation. The smaller the \(\sigma\) the more likely the image will be to pick up noise.

Single 2D Edge Detection Filter

The 1D case of the second derivative of the Gaussian was an ‘inverted Mexican hat’ operator. The ‘zero crossings’ of the second derivative correspond to the edges.

There’s a problem! How does this extend to two dimensions? If you’re calculating second partial derivatives then you have \(f_{xx}\), \(f_{xy}\), \(f_{yy}\), and \(f_{yx}\). Which do you use?

Some of them; Use the Laplacian.

\[ \nabla ^ 2 h = \frac{ \partial ^ 2 f }{ \partial x ^ 2 } + \frac{ \partial ^ 2 f }{ \partial y ^ 2 } \]

This is a single operator that can find edges on both the X and the Y. Yay! Do the Mexican sombrero dance.

Edge Demo

import cv2

# Helper functions

from cs6476helpers import imshow
from cs6476helpers import surf
from cs6476helpers import norm
from cs6476helpers import LoG

# Load in the image and display it

img = cv2.imread("lena.png")

imshow(img,title="Lena")

# Make the image into a grayscale image and display it

img2 = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

imshow(img2,title="Boring Lena")

# Make a Gaussian filter and plot as surface

filter_size  = 10

filter_sigma = 3

gaussian = cv2.getGaussianKernel(filter_size,filter_sigma)

gaussian = gaussian * gaussian.T

surf(gaussian)

# Apply the filter and show the smoothed image

lena_smoothed = cv2.filter2D(img2,-1,gaussian)

imshow(lena_smoothed,title="Blurry Lena")

# Method one, shift differential. Shift the image to L, then R, then show the difference.

lena_left = np.roll(lena_smoothed,-4,1)

lena_right = np.roll(lena_smoothed,4,1)

imshow(norm(lena_left*1.+lena_right*1.))

# Method two, Canny filter on both the unsmoothed and smoothed images

canny_lena = cv2.Canny(img2, 60, 110)

imshow(canny_lena)

uncanny_lena = cv2.Canny(lena_smoothed, 60, 110)

imshow(uncanny_lena)

# Method three, Laplacian of Gaussian

laplacian = LoG(4, 1.)

surf(laplacian)

laplacian_edges = cv2.filter2D(img2,-1,laplacian)

imshow(norm(laplacian_edges),title="Laplacian Lena")
Original Lena

Original Lena

Monochrome Lena

Monochrome Lena

Gaussian Surface

Gaussian Surface

Fuzzy Lena

Fuzzy Lena

Shifted Lena

Shifted Lena

Laplacian of Gaussian

Laplacian of Gaussian

Canny Lena

Canny Lena

Laplacian Lena

Laplacian Lena

Uncanny Lena

Uncanny Lena


  1. https://en.wikipedia.org/wiki/Hysteresis